Robot, robot system, control device, and control method

ABSTRACT

Provided is a robot including a hand and a control unit that operates the hand. The control unit rotates a first object around a predetermined position of the first object with the hand and moves the first object with respect to a second object, based on a captured image including the hand and the first object.

CROSS-REFERENCE TO RELATED APPLICATIONS

Priority is claimed on Japanese Patent Application No. 2014-045752,filed Mar. 7, 2014, the content of which is incorporated herein byreference.

BACKGROUND OF THE INVENTION

Field of the Invention

The present invention relates to a robot, a robot system, a controldevice, and a control method.

Description of the Related Art

A technology for performing compliant motion control of a robot based oninformation acquired by a force sensor, which is information on a forceacting on a gripping unit of the robot or a manipulation target grippedby the gripping unit, and a captured image imaged by an imaging unit hasbeen researched and developed.

In connection with this, for example, a control device that controls arobot to maintain a motion characteristic set for each axis of compliantmotion control in an initial state and perform good assembly work evenwhen a posture for an manipulation target is changed during the work, bysequentially acquiring directions of coordinate axes of the compliantmotion control defined for the manipulation target from a captured imagewhen the robot is caused to perform assembly work of assembling themanipulation target and an assembly target into a predetermined statehas been known. Also, in such a control device, a method of controllingthe robot to perform good assembly work by setting a rotation centerwhen the manipulation target rotates to a predetermined position of themanipulation target depending on a rotation moment applied to themanipulation target has been known (see Proceedings of the 2001 IEEE/RSJInternational Conference on Intelligent Robots and Systems, pp.1477-1482, 2001).

However, in the method of the related art, when a position of therotation center is changed with a relative position between the grippingunit and the manipulation target as the manipulation target comes incontact with the assembly target during work, the change in the positionof the rotation center cannot be detected and good assembly work may notbe performed.

SUMMARY OF THE INVENTION

Therefore, the present invention has been made in view of the problemsof the method in the related art, and provides a robot, a robot system,a control device, and a control method capable of performing goodassembly work.

According to a first aspect of the present invention, a robot includes:a hand; and a control unit that operates the hand, wherein the controlunit rotates a first object around a predetermined position of the firstobject and relatively moves the first object with respect to a secondobject with the hand, based on a captured image including the hand andthe first object.

Through this configuration, the robot rotates the first object around apredetermined position of the first object and relatively moves thefirst object with respect to a second object with the hand, based on thecaptured image including the hand and the first object. Therefore, therobot can perform good assembly work.

According to a second aspect of the present invention, the predeterminedposition may be a coordinate origin that moves with the first object,and the control unit may translate the first object in addition torotating the first object.

Through this configuration, the robot uses a rotation center in the caseof rotation as the coordinate origin moving with the first object, andrelatively moves the first object with respect to the second objectthrough rotation and translation. Therefore, the robot can perform goodassembly work.

According to a third aspect of the present invention, the control unitmay perform visual servo control based on the captured image.

Through this configuration, the robot performs visual servo controlbased on the captured image. Therefore, the robot can relatively movethe first object with respect to the second object, and as a result, canperform good assembly work with high precision.

According to a fourth aspect of the present invention, the control unitmay perform compliant motion control according to a motioncharacteristic set in the predetermined position and each axialdirection.

Through this configuration, the robot performs compliant motion controlaccording to the motion characteristic set in the predetermined positionand each axial direction. Therefore, the robot can assemble the firstobject with respect to the second object without damaging the secondobject.

According to a fifth aspect of the present invention, the control unitmay derive a relative positional relationship between a position set inthe hand and a position set in the first object based on the capturedimage, and update the predetermined position based on the derivedpositional relationship.

Through this configuration, the robot derives the relative positionalrelationship between the position set in the hand and the position setin the first object based on the captured image, and updates thepredetermined position based on the derived positional relationship.Therefore, even when the positional relationship between the hand andthe predetermined position is shifted due to an external force, therobot can relatively move the first object with respect to the secondobject around the shifted predetermined position and, as a result, canperform good assembly work.

According to a sixth aspect of the present invention, the control unitmay update the predetermined position based on the derived positionalrelationship, and a relative positional relationship between theposition set in the first object and the predetermined position.

Through this configuration, the robot derives the relative positionalrelationship between the position set in the hand and the position setin the first object, and updates the predetermined position based on thederived positional relationship, and the relative positionalrelationship between the position set in the first object and thepredetermined position. Therefore, even when the positional relationshipbetween the hand and the predetermined position is shifted due to anexternal force, the robot can indirectly recognize the shiftedpredetermined position from the relative positional relationship betweenthe position set in the hand and the predetermined position through theposition set in the first object and, as a result, can relatively movethe first object with respect to the second object around the shiftedpredetermined position.

According to a seventh aspect of the present invention, the robot mayinclude a marker indicating a position of a force sensor that detects anexternal force acting on a hand, the captured image may further includethe marker, a position set in the hand may be represented by a positionof the force sensor in the robot, and the control unit may derive arelative positional relationship between a position set in the hand andthe predetermined position based on the position of the marker detectedfrom the captured image, and update the predetermined position based onthe derived relative positional relationship between the position set inthe hand and the predetermined position.

Through this configuration, the robot derives a relative positionalrelationship between the position set in the hand and the predeterminedposition based on the position of the marker indicating the position ofthe force sensor detected from the captured image, and updates thepredetermined position based on the derived relative positionalrelationship between the position set in the hand and the predeterminedposition. Therefore, even when the force sensor is covered with a memberof an arm unit of the robot, the robot can recognize the position of thefirst object using the position of the marker indicating the forcesensor as a mark and, as a result, can perform good assembly work.

According to an eighth aspect of the present invention, the robot mayinclude a force sensor that detects an external force acting on thehand, a position set in the hand may be represented by a position of theforce sensor, and the control unit may derive a relative positionalrelationship between the position set in the hand and the predeterminedposition based on a position and a posture in an initial state of theforce sensor, and updates the predetermined position based on thederived relative positional relationship between the position set in thehand and the predetermined position may be used.

Through this configuration, the robot derives the relative positionalrelationship between the position set in the hand and the predeterminedposition based on the position and the posture in an initial state ofthe force sensor, and updates the predetermined position based on thederived relative positional relationship between the position set in thehand and the predetermined position. Therefore, even when an arm unit ofthe robot including the force sensor is shifted from an angle of view ofthe imaging unit, the robot can recognize the position of the firstobject and, as a result, can perform good assembly work.

According to a ninth aspect of the present invention, a robot systemincludes: a robot including a hand; and an imaging unit that images thehand and a first object, wherein the robot rotates the first objectaround a predetermined position of the first object and relatively movesthe first object with respect to a second object based on a capturedimage including the hand and the first object captured by the imagingunit.

Through this configuration, the robot system rotates the first objectaround the predetermined position of the first object and relativelymoves the first object with respect to the second object based on thecaptured image including the hand and the first object. Therefore, therobot system can perform good assembly work using the robot.

According to a tenth aspect of the present invention, a control devicefor operating a robot includes a hand, wherein the control device causesthe robot to rotate a first object around a predetermined position ofthe first object and relatively move the first object with respect to asecond object, based on a captured image including the hand and thefirst object.

Through this configuration, the control device causes the robot torotate a first object around the predetermined position of the firstobject and relatively move the first object with respect to the secondobject, based on the captured image including the hand and the firstobject. Therefore, the control device can perform good assembly workusing the robot.

According to an eleventh aspect of the present invention, a controlmethod for operating a robot including a hand, includes: acquiring acaptured image including the hand and a first object; and rotating thefirst object around a predetermined position of the first object andrelatively moving the first object with respect to a second object basedon the captured image.

Through this configuration, the method includes rotating the firstobject around the predetermined position of the first object andrelatively moving the first object with respect to a second object basedon the captured image. Therefore, the control method can perform goodassembly work.

Thus, the robot, the robot system, the control device, and the controlmethod rotates the first object around the predetermined position of thefirst object and relatively moves the first object with respect to thesecond object with the hand, based on the captured image including thehand and the first object. Therefore, the robot, the robot system, thecontrol device and the control method can perform good assembly work.

BRIEF DESCRIPTION OF THE DRAWINGS

The above features and advantages of the present invention will be moreapparent from the following description of certain preferred embodimentstaken in conjunction with the accompanying drawings.

FIG. 1 is a configuration diagram illustrating an example of the robotsystem 1 according to a first embodiment.

FIG. 2 is a diagram illustrating an example of a coordinate system usedin the robot system 1.

FIG. 3 is a diagram illustrating an example of a hardware configurationof a control device 30.

FIG. 4 is a diagram illustrating an example of a functionalconfiguration of the control device 30.

FIG. 5 is a flowchart illustrating an example of a process flow in whichthe control device 30 controls a robot 20 so as to assemble amanipulation target M and an assembly target O.

FIG. 6 is a configuration diagram illustrating an example of a robotsystem 2 according to a second embodiment.

FIGS. 7A to 7D are diagrams schematically illustrating an example ofpredetermined work performed by a robot system 1 according to a thirdembodiment.

FIG. 8 is a flowchart illustrating an example of a process flow in whicha control unit 36 of a control device 30 operates a robot 20 to tightena bolt O1 with a wrench M1.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS First Embodiment

Hereinafter, a first embodiment of the present invention will bedescribed with reference to the drawings. FIG. 1 is a configurationdiagram illustrating an example of a robot system 1 according to thefirst embodiment. The robot system 1 includes an imaging unit 10, arobot 20, and a control device 30.

The imaging unit 10 is, for example, a camera including a CCD (ChargeCoupled Device) or a CMOS (Complementary Metal Oxide Semiconductor)which is an imaging element that converts condensed light into anelectrical signal. Further, the imaging unit 10 is a stereo cameraincluding two cameras, but may include, for example, three or morecameras or may have a configuration in which a two-dimensional image iscaptured by one camera.

The imaging unit 10 is connected to the control device 30 via a cable sothat the imaging unit 10 can communicate with the control device 30. Forexample, wired communication via the cable is performed according to astandard such as Ethernet (registered trademark), a USB (UniversalSerial Bus), or the like. Further, the imaging unit 10 and the controldevice 30 may be connected through wireless communication performedaccording to a communication standard such as Wi-Fi (registeredtrademark).

The imaging unit 10 is installed in a position in which a rangeincluding the robot 20, a manipulation target M gripped by a grippingunit HND of the robot 20, and an assembly target O into which themanipulation target M is assembled by the robot 20 can be imaged. Themanipulation target M is, for example, a member assembled into theassembly target O in a predetermined state and forming one industrialproduct. Further, in the following description, the manipulation targetM is assumed to have been gripped by the gripping unit HND of the robot20 in advance.

The assembly target O is installed in a position apart from the robot 20by a jig or the like in advance in a range in which the manipulationtarget M can be assembled using the robot 20, as illustrated in FIG. 1.

For example, the robot 20 is a single arm robot including the grippingunit HND (an end effector), a force sensor 22, an arm unit ARM (amanipulator), and a plurality of actuators, which are not illustrated.Further, the robot system 1 may have a configuration in which a dual-armrobot is included, in place of a configuration in which a single armrobot is included. An embodiment of the configuration in which the robotsystem 1 includes a dual-arm robot will be described in the secondembodiment.

The arm of the robot 20 is a 6-axis vertical multi-joint type, and canperform an operation at a degree of freedom of six axes through anoperation in which a support stand, the arm unit ARM, and the grippingunit HND cooperate using the actuators. Further, the arm of the robot 20may operate at 5 degrees of freedom (5 axes) or less, and may operate at7 degrees of freedom (7 axes) or more. Hereinafter, an operation of therobot 20 performed by the arm including the gripping unit HND and thearm unit ARM will be described. Further, the gripping unit HND is anexample of a hand.

The robot 20 is connected to the control device 30, for example, via acable so that the robot can communicate with the control device 30. Forexample, wired communication through the cable is performed according toa standard such as Ethernet (registered trademark) or a USB. Further,the robot 20 and the control device 30 may be connected through wirelesscommunication performed according to a communication standard such asWi-Fi (registered trademark). Further, in the robot system 1, aconfiguration in which the robot 20 is connected to the control device30 installed outside the robot 20 as illustrated in FIG. 1 is adopted.However, in place of this configuration, a configuration in which thecontrol device 30 is built in the robot 20 may be adopted.

The gripping unit HND of the robot 20 includes a claw unit which cangrip an object.

The force sensor 22 is included between the gripping unit HND and thearm unit ARM of the robot 20 and detects a force or a moment acting onthe gripping unit HND (or the manipulation target M gripped by thegripping unit HND). The force sensor 22 outputs information indicatingthe detected force or moment to the control device 30 throughcommunication. For example, the information indicating the force or themoment detected by the force sensor 22 is used for, for example,compliant motion control of the robot 20 by the control device 30.

The robot 20 acquires a control signal based on a relative positionalrelationship among the manipulation target M, the assembly target O, andthe force sensor 22 from the control device 30, and performspredetermined work on the manipulation target M based on the acquiredcontrol signal. The predetermined work is, for example, work of movingthe manipulation target M gripped by the gripping unit HND of the robot20 from a current position, and assembling the manipulation target Minto the assembly target O.

The control device 30 controls the robot 20 to perform predeterminedwork. More specifically, the control device 30 derives a relativepositional relationship of the manipulation target M, the assemblytarget O, and the force sensor 22 based on a captured image captured bythe imaging unit 10, which is a captured image including an imaged rangein which the robot 20, the manipulation target M gripped by the grippingunit HND of the robot 20, and the assembly target O into which themanipulation target M is assembled by the robot 20 are included.

Also, the control device 30 controls the robot 20 to performpredetermined work based on the derived relative positionalrelationship. That is, the control device 30 controls the robot 20 sothat the manipulation target M is assembled into the assembly target Oby relatively moving the manipulation target M with respect to theassembly target O using the gripping unit HND and the arm unit ARM.

Further, the control device 30 may control the robot 20 so that therobot 20 rotates the manipulation target M gripped by the gripping unitHND when relatively moving the manipulation target M with respect to theassembly target O. In this case, the control device 30 controls therobot 20 to rotate the manipulation target M around a predeterminedposition (hereinafter referred to as a rotation center position) set inthe manipulation target M. Further, a posture of the manipulation targetM in the rotation center position is hereinafter referred to as arotation center posture.

In this control, even when a relative position and a relative posture ofthe manipulation target M and the force sensor 22 (a relative positionalrelationship) vary, the control device 30 detects a rotation centerposition and a rotation center posture after the relative positionalrelationship varies from the captured image captured by the imaging unit10. Based on the detected rotation center position and the detectedrotation center posture, the control device 30 always rotates themanipulation target M based on the rotation center posture around therotation center position set in the manipulation target M. Further, therotation center position is set to any position on the manipulationtarget M by a user. Further, while the rotation center posture is aposture of the manipulation target M in the rotation center position,the rotation center posture may not match the posture of themanipulation target M as long as the rotation center posture isassociated with the posture of the manipulation target M.

Here, coordinate systems used in the robot system 1 will be describedwith reference to FIG. 2. FIG. 2 is a diagram illustrating coordinatesystems used in the robot system 1. Further, a letter after “_” isdescribed to indicate a subscript of a letter before “_” in thefollowing description. The control device 30 of the robot system 1performs a control process so that the robot 20 performs predeterminedwork using seven three-dimensional Cartesian coordinate systems, thatis, an imaging unit coordinate system Σ_c, a work coordinate system Σ_w,a tool coordinate system Σ_t, a gravity center coordinate system Σ_g, amanipulation target coordinate system Σ_m, an external force coordinatesystem Σ_e, and an assembly target coordinate system Σ_o, as illustratedin FIG. 2. Origins of these seven coordinate systems and directions ofthe coordinate axes are set (stored or registered) in the control device30 by the user.

Each of the origins of the seven coordinate systems is set to move witha target X so as to represent a position of the target X (in this case,the imaging unit 10, a support stand of the robot 20, the force sensor22, a center of gravity of the manipulation target M, a rotation centerof the manipulation target M, the manipulation target M, or the assemblytarget O). Further, each of the directions of the coordinate axes of theseven coordinate systems is set to move with a change in a posture ofthe target X so as to represent the posture of the target X. Further,the user may associate the position of the target X with the position ofthe origin of the coordinate system, and may set the position of theorigin of the coordinate system to be set for the target X and thedirection of the coordinate axis to an arbitrary position and directionon the assumption that a slope of the target X and the direction of thecoordinate system can be associated.

The imaging unit coordinate system Σ_c is a coordinate systemrepresenting a position (for example, a position determined in advanceon an imaging element is the origin) and a posture of the imaging unit10.

The work coordinate system Σ_w is a coordinate system representing aposition and a posture of the support stand of the robot 20.

The tool coordinate system Σ_t is a coordinate system set in a position(for example, a position of a marker indicating a center of gravity ofthe force sensor 22 or a position of the force sensor 22 is the origin)or a posture of the force sensor 22. Further, in this embodiment, thetool coordinate system Σ_t represents a position and a posture of theforce sensor 22 and a position and a posture of the gripping unit HND soas to simplify description. Generally, a sensor coordinate systemrepresenting a position and a posture of the force sensor 22 and a hand(gripping unit) coordinate system representing a position and a postureof the gripping unit HND do not match, and the control device 30calculates a relative positional relationship between the workcoordinate system and the hand coordinate system using the relativepositional relationship between the work coordinate system and thesensor coordinate system and the relative positional relationshipbetween the sensor coordinate system and the hand coordinate system, andperforms control of the robot 20 based on the calculated relativepositional relationship between the work coordinate system and the handcoordinate system.

The gravity center coordinate system Σ_g is a coordinate systemrepresenting a position and a posture of the center of gravity of themanipulation target M.

The manipulation target coordinate system Σ_m is a coordinate systemrepresenting a position and a posture of the manipulation target M (forexample, a position on the manipulation target M most apart from thegripping unit HND in an initial state).

The external force coordinate system Σ_e is a coordinate system thatdefines an external force and external moment acting on the target.Further, in this disclosure, a coordinate system that defines a motionbased on compliant motion control is caused to match the external forcecoordinate system Σ_e. That is, rotation motion based on compliantmotion control is represented by rotation around the origin of theexternal force coordinate system Σ_e based on the posture of theexternal force coordinate system Σ_e. However, the systems may be notmatched or may be arbitrarily arranged by the user. Further,hereinafter, an external force (that is, an external force detected bythe force sensor) acting on the target is simply referred to as a force,and an external moment (that is, a moment detected by the force sensor)is referred to as a moment as long as it is not necessary to distinguishthem. As described above, the rotation center position can be set to anarbitrary position by the user, but is assumed to be set to apredetermined position on the manipulation target M in this embodiment.

The assembly target coordinate system Σ_o is a coordinate systemrepresenting a position and a posture of the assembly target O (forexample, a position on the assembly target O nearest to the manipulationtarget M).

In the following description, a position of an origin of the coordinatesystem b in the coordinate system a is assumed to be a position of thetarget X in which the coordinate system b is set since the coordinatesystem moves with the target X. For example, the position of the originof the manipulation target coordinate system Σ_m in the work coordinatesystem Σ_w is referred to as a position of the manipulation target M inthe work coordinate system Σ_w. Similarly, the posture of the coordinatesystem b in the coordinate system a will be described as a posture ofthe target X in which the coordinate system b is set. For example, theposture of the manipulation target coordinate system Σ_m in the workcoordinate system Σ_w is referred to as a posture of the manipulationtarget M in the work coordinate system Σ_w.

Here, a notation of equations used to describe a process performed bythe control device 30 is shown prior to a more concrete description.First, in the following description, a letter after “^” is described toindicate a superscript of a letter before “^”. Further, a first letterwith “(→)” is described to be a vector. Further, a first letter with“(^)” is described to be a matrix.

Under this notation, in the coordinate system a, a vector representingthe position of the target X in which the coordinate system b is set isrepresented as a position vector p_b^a(→). The position vector p_b^a(→)is defined by an x coordinate x^b, a y coordinate y^b, and az-coordinate z^b of the target X in the coordinate system b, as shown inEquation (1) below.{right arrow over (p_(b) ^(a))}=[x_(b) ^(a)y_(b) ^(a)z_(b) ^(a)]  (1)

Similar to the notation of the position vector, a vector representingthe posture of the target X in which the coordinate system b is set inthe coordinate system a is represented as a posture vector o_b^a(→). Theposture vector o_b^a(→) is represented as Equation (2) below usingEulerian angles (α_b^a, β_b^a, and γ_b^a) as components, as shown inEquation (2) below.{right arrow over (o_(b) ^(a))}=[α_(b) ^(a)β_(b) ^(a)γ_(b) ^(a)]  (2)

Here, the Eulerian angles are defined as angles rotated around a z-axis,a y-axis and an x-axis of the coordinate system a to cause the x-axis,the y-axis and the z-axis of the coordinate system a to match an x-axis,an y-axis and a z-axis of the coordinate system b, and are representedas γ_b^a, β_b^a and α_b^a.

A rotation matrix when the position and the posture of the target Xrepresented by the posture of the coordinate system b are rotated to theposition and the posture of the target X represented by the posture ofthe coordinate system a is represented as a rotation matrix R_b^a(^).Further, there is a relationship between the above-described Eulerianangles and the rotation matrix in Equation (3) shown below.

$\begin{matrix}{\mspace{79mu}{{{\overset{\;\rightarrow}{o}\left( \hat{R} \right)} = \left\lbrack \;{{{atan}\left( {R_{33}/{- R_{23}}} \right)}\;{{asin}\left( R_{13} \right)}\;{{atan}\left( {{- R_{32}}/R_{13}} \right)}} \right\rbrack}{{\hat{R}\left( \overset{\;\rightarrow}{o} \right)} = \begin{bmatrix}{\cos\;{\beta cos\gamma}} & {{- \cos}\;{\beta sin\gamma}} & {\sin\;\beta} \\{{\sin\;{\alpha sin\beta cos\gamma}} +} & {{{- \sin}\;{\alpha sin\beta sin\gamma}} +} & {{- \sin}\;{\alpha cos\beta}} \\{\;{\cos\;{\alpha sin\gamma}}} & {\;{\cos\;{\alpha cos\gamma}}} & \; \\{{{- \cos}\;{\alpha sin\beta cos\gamma}} +} & {{\cos\;{\alpha sin\beta sin\gamma}} +} & {\cos\;{\alpha cos\beta}} \\{\;{\sin\;{\alpha sin\gamma}}} & {\sin\;{\alpha cos\gamma}} & \;\end{bmatrix}}}} & (3)\end{matrix}$

Here, for the vectors shown in Equations (1) to (3) described above, atop and a bottom of the subscript can be replaced in conversionequations (4) to (6) below.{right arrow over (o)}_(a) ^(b)={right arrow over (o)}([{circumflex over(R)}({right arrow over (o)}_(b) ^(a))]^(T))  (4){right arrow over (p)}_(a) ^(b)=[{circumflex over (R)}_(b)^(a)]^(T)(−{right arrow over (p)}_(b) ^(a))  (5){right arrow over (R)}_(a) ^(b)=[{circumflex over (R)}_(b)^(a)]^(T)  (6)

Here, [R_b^a(^)]^T indicates a transposed matrix of R_a^b. That is, asubscript of the rotation matrix can be replaced by transposing therotation matrix, and a subscript of the vector can be replaced by therotation matrix. Further, hereinafter, a position vector p_b^a(→)indicating the position of the target X in which the coordinate system bis set in the coordinate system a is simply referred to as a position ofthe target X in the coordinate system a, except for a case in whichthere is a need. For example, a position vector p_o^c(→) indicating theposition of the origin of the assembly target coordinate system Σ_o setin the assembly target O represented in the imaging unit coordinatesystem Σ_c is simply referred to as a position of the assembly target Oin the imaging unit coordinate system Σ_c.

Similar to the case of the position, hereinafter, a posture vectoro_b^a(→) indicating the posture of the target X in which the coordinatesystem b is set in the coordinate system a is simply referred to as aposture of the target X in the coordinate system a except for a case inwhich there is a need. For example, a posture vector o_o^c(→) indicatingthe posture of the assembly target coordinate system Σ_o set in theassembly target O represented in the imaging unit coordinate system Σ_cis simply referred to as a posture of the assembly target O in theimaging unit coordinate system Σ_c.

Next, a hardware configuration of the control device 30 will bedescribed with reference to FIG. 3. FIG. 3 is a diagram illustrating anexample of the hardware configuration of the control device 30. Thecontrol device 30 includes, for example, a CPU (Central Processing Unit)31, a storage unit 32, an input reception unit 33, and a communicationunit 34, and performs communication with the imaging unit 10, the robot20 or the like via the communication unit 34. These components areconnected via a bus so that the components can communicate with eachother. The CPU 31 executes various programs stored in the storage unit32.

The storage unit 32 includes, for example, an HDD (Hard Disk Drive), anSSD (Solid State Drive), an EEPROM (Electrically Erasable ProgrammableRead-Only Memory), a ROM (Read-Only Memory), or a RAM (Random AccessMemory), and stores various pieces of information, images, and programsprocessed by the control device 30. Further, the storage unit 32 may bean external storage device connected by, for example, a digital inputand output port such as a USB, rather than a storage unit built in thecontrol device 30.

The input reception unit 33 is, for example, a keyboard, a mouse, atouch pad, or another input device. Further, the input reception unit 33may function as a display unit or may be configured as a touch panel.

The communication unit 34 includes, for example, a digital input andoutput port such as a USB, or an Ethernet port.

Next, a functional configuration of the control device 30 will bedescribed with reference to FIG. 4. FIG. 4 is a diagram illustrating anexample of a functional configuration of the control device 30. Thecontrol device 30 includes a storage unit 32, an input reception unit33, an image acquisition unit 35, and a control unit 36. For example,some or all of these functional units are realized by the CPU 31, whichexecutes various programs stored in the storage unit 32. Further, someor all of these functional units may be hardware functional units suchas an LSI (Large Scale Integration) or an ASIC (Application SpecificIntegrated Circuit).

The control device 30 relatively moves the manipulation target M withrespect to the assembly target O so that the manipulation target M andthe assembly target O are assembled without damaging the manipulationtarget M by combining compliant motion control into a loop of visualservo. More specifically, the control device 30 controls the robot 20 sothat the manipulation target M changes into the same state as that of atemplate image (for example, a CAD (Computer Aided Design) image in astate in which the manipulation target M and the assembly target O areassembled) stored in the storage unit 32 in advance using a visual servotechnology.

In this case, when the control device 30 detects a relative positionalrelationship between the manipulation target M and the force sensor 22in the captured image captured by the imaging unit 10, and sequentiallydetermines the rotation center position and the rotation center posturedefining its operation when controlling the robot 20 through compliantmotion control based on the detected relative positional relationship.Also, when the control device 30 causes the robot 20 to perform anoperation of rotating the manipulation target M, the control device 30controls the robot 20 to rotate the manipulation target M based on therotation center posture around the rotation center position. Further,when the control device 30 causes the robot 20 to translate themanipulation target M, the control device 30 controls the robot 20 totranslate the manipulation target M based on the rotation centerposture.

Further, the control device 30 includes a time measuring unit, which isnot illustrated, acquires the captured image captured by the imagingunit 10 from the image acquisition unit 35 at a timing measured by thetime measuring unit, and acquires information indicating the force andthe moment detected by the force sensor 22 from the force sensor 22 ofthe robot 20 at the same timing.

The image acquisition unit 35 acquires the captured image captured bythe imaging unit 10. The image acquisition unit 35 outputs the acquiredcaptured image to the control unit 36.

The control unit 36 includes a target derivation unit 39, a rotationcenter position calculation unit 41, an external force calculation unit43, a load relaxation operation amount calculation unit 45, a movementoperation amount calculation unit 46, an operation end determinationunit 47, and a robot control unit 49.

The target derivation unit 39 detects a position and a posture of theassembly target O in the imaging unit coordinate system Σ_c from thecaptured image acquired by the image acquisition unit 35.

Further, the target derivation unit 39 derives a position and a postureof the manipulation target M in the imaging unit coordinate system Σ_cafter assembly completion, based on the position and the posture of theassembly target O in the imaging unit coordinate system Σ_c. Whenperforming this derivation, the target derivation unit 39 detects theposition and the posture of the manipulation target M after the assemblycompletion based on the above-described template image. Hereinafter, theposition and the posture of the manipulation target M in the imagingunit coordinate system Σ_c after the assembly completion derived by thetarget derivation unit 39 are referred to as a target position and atarget posture.

The rotation center position calculation unit 41 detects a currentposition and current posture of the manipulation target M in the imagingunit coordinate system Σ_c from the captured image acquired by the imageacquisition unit 35. Further, the rotation center position calculationunit 41 detects the position and the posture of the imaging unit 10 inthe tool coordinate system Σ_t. Further, the rotation center positioncalculation unit 41 calculates a rotation center position and a rotationcenter posture in the tool coordinate system Σ_t based on the detectedcurrent position and posture of the manipulation target M in the imagingunit coordinate system Σ_c, the detected position and posture of theimaging unit 10 in the tool coordinate system Σ_t, and the rotationcenter position and the rotation center posture in the manipulationtarget coordinate system Σ_m set in advance by the user.

Further, the rotation center position calculation unit 41 calculates aposition and a posture of the force sensor 22 in the work coordinatesystem Σ_w using forward kinematics. Further, the rotation centerposition calculation unit 41 calculates a position and a posture of thesupport stand of the robot 20 in the external force coordinate systemΣ_e based on the calculated rotation center position and rotation centerposture in the tool coordinate system Σ_t, and the position and theposture of the force sensor 22 in the work coordinate system Σ_w.

The external force calculation unit 43 calculates the force and themoment acting on the gripping unit HND acquired from the force sensor22, which are a force and moment represented in the tool coordinatesystem Σ_t, as a force and moment in the external force coordinatesystem Σ_e, based on the position and the posture of the support standof the robot 20 in the external force coordinate system Σ_e calculatedby the rotation center position calculation unit 41.

The load relaxation operation amount calculation unit 45 calculates arelaxation operation amount for moving the manipulation target M, whichis a relaxation operation amount in the external force coordinate systemΣ_e to relax the force acting on the manipulation target M gripped bythe gripping unit HND of the robot 20, based on the force and the momentcalculated by the external force calculation unit 43, which are a forceand moment represented in the external force coordinate system Σ_e.Here, the relaxation operation amount is a small movement amount fortranslating the manipulation target M in a direction in which thedetected force acts (hereinafter referred to as a small relaxationmovement amount), and a small rotation amount for rotating themanipulation target M in a direction of the detected moment (hereinafterreferred to as a small relaxation rotation amount). The load relaxationoperation amount calculation unit 45 calculates a relaxation operationamount in the work coordinate system Σ_w based on the calculatedrelaxation movement amount in the external force coordinate system Σ_e.

The movement operation amount calculation unit 46 calculates, in theimaging unit coordinate system Σ_c, a target operation amount by whichthe manipulation target M is moved by the gripping unit HND so that theposition and posture of the manipulation target M match the targetposition and posture calculated by target derivation unit 39. The targetoperation amount is a small movement amount for translating themanipulation target M to the target position (hereinafter referred to asa small target movement amount) and a small rotation amount for rotatingthe manipulation target M to the target posture (hereinafter referred toas a small target rotation amount). The movement operation amountcalculation unit 46 calculates a target operation amount in the workcoordinate system Σ_w based on the calculated target operation amount inthe imaging unit coordinate system Σ_c.

The operation end determination unit 47 determines whether work in whichthe robot 20 assembles the manipulation target M into the assemblytarget O ends based on the relaxation operation amount in the workcoordinate system Σ_w calculated by the load relaxation operation amountcalculation unit 45 and the target operation amount in the workcoordinate system Σ_w calculated by the movement operation amountcalculation unit 46. Further, the operation end determination unit 47may determine whether the work in which the robot 20 assembles themanipulation target M into the assembly target O ends based on only thetarget operation amount in the work coordinate system Σ_w calculated bythe movement operation amount calculation unit 46.

The robot control unit 49 calculates a position and a posture in whichthe force sensor 22 is to move in the work coordinate system Σ_w basedon the relaxation operation amount in the work coordinate system Σ_wcalculated by the load relaxation operation amount calculation unit 45and the target operation amount in the work coordinate system Σ_wcalculated by the movement operation amount calculation unit 46. Also,the robot control unit 49 controls the robot 20 so that the position andthe posture of the force sensor 22 match the calculated position andposture in which the force sensor 22 is to move in the work coordinatesystem Σ_w.

Hereinafter, a process in which the control device 30 operates the robot20 so that the manipulation target M and the assembly target O areassembled will be described with reference to FIG. 5. FIG. 5 is aflowchart illustrating an example of a process flow in which the controldevice 30 operates the robot 20 so that the manipulation target M andthe assembly target O are assembled.

First, the control device 30 sets a position and a posture input fromthe user via the input reception unit 33, which is a position p_o^m(→)and a posture o_o^m(→) of the assembly target O in the manipulationtarget coordinate system Σ_m in a state in which assembly of themanipulation target M and the assembly target O is completed (stepS100).

Then, the control device 30 sets a rotation center position and arotation center posture input from the user via the input reception unit33, which is a rotation center position p_e^m(→) and a rotation centerposture o_e^m(→) in the manipulation target coordinate system Σ_m (stepS110).

Steps S100 to S110 are initial settings for the control device 30performed by the user. Now, it is assumed that, after step S110, imagingby the imaging unit 10 is started and the control unit 36 acquires thecaptured image from the image acquisition unit 35.

After the control unit 36 acquires the captured image, the targetderivation unit 39 of the control device 30 detects a position p_o^c(→)and a posture o_o^c(→) of the assembly target O in the imaging unitcoordinate system Σ_c based on the captured image acquired by the imageacquisition unit 35 (step S120).

Then, the target derivation unit 39 calculates a target position and atarget posture of the manipulation target M in a state in which assemblyof the manipulation target M and the assembly target O is completed,which is the target position p_m^c(d)(→) and the target postureo_m^c(d)(→) of the manipulation target M in the imaging unit coordinatesystem Σ_c, using Equation (7) shown below, based on the positionp_o^m(→) and the posture o_o^c(→) of the assembly target O in themanipulation target coordinate system Σ_m in a state in which assemblyof the manipulation target M and the assembly target O is completed,which are set in step S100, and the position p_o^c(→) and the postureo_o^c(→) of the assembly target O in the imaging unit coordinate systemΣ_c detected in step S120 (step S130){right arrow over (p)}_(m) ^(c)(d)={right arrow over (p)}_(o)^(c)+{circumflex over (R)}_(o) ^(c){right arrow over (p)}_(m) ^(o){circumflex over (R)}_(m) ^(c)(d)={circumflex over (R)}_(o)^(c){circumflex over (R)}_(m) ^(o)  (7)

Here, “(d)” of the target position p_m^c(d)(→) and the target postureo_m^c(d)(→) of the manipulation target M in the imaging unit coordinatesystem Σ_c is a label added to distinguish between the position p_m^c(→)and the posture o_m^c(→) of the manipulation target M in the imagingunit coordinate system Σ_c detected by the rotation center positioncalculation unit 41 in step S140.

The process from step S120 to step S130 is a process in which thecontrol unit 36 indirectly calculates a relative position and a relativeposture of the manipulation target M in a state in which assembly of themanipulation target M and the assembly target O is completed and theimaging unit 10, based on the relative position and the relative postureof the imaging unit 10 and the assembly target O, and the relativepositional relationship (the position and the posture) of themanipulation target M and the assembly target O in a state in whichassembly of the manipulation target M and the assembly target O iscompleted.

Then, the rotation center position calculation unit 41 detects theposition p_m^c(→) and the posture o_m^c(→) of the manipulation target Min the imaging unit coordinate system Σ_c from the acquired capturedimage (step S140).

Then, the rotation center position calculation unit 41 detects aposition p_c^t(→) and a posture o_c^t (→) of the imaging unit 10 in thetool coordinate system Σ_t from the acquired captured image (step S150).

Then, the rotation center position calculation unit 41 calculates aposition p_e^t(→) and a posture o_e^t(→) of the rotation center in thetool coordinate system Σ_t using Equation (8) shown below, based on theposition p_m^c(→) and the posture o_m^c(→) of the manipulation target Min the imaging unit coordinate system Σ_c calculated in step S140 andthe position p_c^t(→) and the posture o_c^t(→) of the imaging unit 10 inthe tool coordinate system Σ_t calculated in step S150 (step S160).{right arrow over (p)}_(e) ^(t)={right arrow over (p)}_(c)^(t)+{circumflex over (R)}_(c) ^(t)({right arrow over (p)}_(m)^(c)+{circumflex over (R)}_(m) ^(c){right arrow over (p)}_(e) ^(m)){circumflex over (R)}_(e) ^(t)={circumflex over (R)}_(c) ^(t){circumflexover (R)}_(m) ^(c){circumflex over (R)}_(e) ^(m)  (8)

Here, the process of step S160 is a process of detecting a change in arelative positional relationship between the force sensor 22 and therotation center position and the rotation center posture describedabove. The fact that the position p_e^t(→) and the posture o_e^t(→) ofthe rotation center in the tool coordinate system Σ_t calculated in stepS160 are different from the values calculated in the previous routineshows that the relative positional relationship between the force sensor22 and the rotation center position and the rotation center posture ischanged due to an external force. By detecting this change, the controlunit 36 can perform control so that the robot 20 always rotates themanipulation target M based on the rotation center posture around therotation center position, based on the rotation center position and therotation center posture with respect to the force sensor 22 after thechange even when the rotation center position and the rotation centerposture with respect to the force sensor 22 are changed due to anexternal force.

Then, the rotation center position calculation unit 41 calculates theposition p_t^w(→) and the posture o_t^w(→) of the force sensor 22 in thework coordinate system Σ_w based on forward kinematics (step S170).

Then, the rotation center position calculation unit 41 calculates theposition p_w^e(→) and the posture o_w^e(→) of the support stand of therobot 20 in the external force coordinate system Σ_e using Equation (9)shown below, based on the rotation center position p_e^t(→) and therotation center posture o_e^t(→) in the tool coordinate system Σ_tcalculated in step S160, and the position p_t^w(→) and the postureo_t^w(→) of the force sensor 22 in the work coordinate system Σ_wcalculated in step S170 (step S180).

$\begin{matrix}{{{\overset{\;\rightarrow}{p}}_{w}^{e} = {{\overset{\;\rightarrow}{p}}_{t}^{e} + {{\hat{R}}_{t}^{e}\;{\overset{\;\rightarrow}{p}}_{w}^{t}}}}\;} & (9) \\{{\hat{R}}_{w}^{e} = {{\hat{R}}_{t}^{e}\;{\hat{R}}_{w}^{t}}} & \;\end{matrix}$

Here, through the process of the following step S190, the control unit36 can calculate the force and the moment acting on the rotation centerposition due to the force and the moment acting on the force sensor 22by calculating the position p_w^e(→) and the posture o_w^e(→) of thesupport stand of the robot 20 in the external force coordinate systemΣ_e through the process of step S180.

Then, the external force calculation unit 43 calculates a force (thatis, a force acting on the rotation center position) f^e(→) and a moment(that is, moment generated in the rotation center position) m^e(→) inthe external force coordinate system Σ_e using a Newton-Euler equation(10) shown below, based on the force and the moment detected by theforce sensor 22, which are the force f^t(→) and the moment m^t(→) in thetool coordinate system Σ_t (step S190).{right arrow over (f)}^(e)={circumflex over (R)}_(w) ^(e){{circumflexover (R)}_(t) ^(w){right arrow over (f)}^(t)−mÊ({right arrow over(g)}−{right arrow over ({umlaut over (p)})}_(c) ^(w))}{right arrow over (m)}^(e)={circumflex over (R)}_(w) ^(e){Î{right arrowover (ö)}_(c) ^(w)+{right arrow over ({dot over (o)})}_(c) ^(w)×Î{rightarrow over ({dot over (o)})}_(c) ^(w)+{circumflex over (R)}_(t)^(w)({right arrow over (m)}^(t)+{right arrow over (p)}_(e) ^(t)×{rightarrow over (f)}^(t)−{right arrow over (p)}_(g) ^(t)×m{circumflex over(R)}_(w) ^(t){right arrow over (g)})}  (10)

Here, the upper equation in Equation (10) described above shows that acomponent f^e(→) of the external force acting on the rotation centerposition of the manipulation target M is represented by a componentobtained by subtracting a component mE(^)g(→) due to gravity and acomponent mE(^)p_c^w(→)(⋅⋅) due to an inertial motion by the arm unitARM of the robot 20 from a component f^t(→) of the force detected by theforce sensor 22. Further, a first letter with “(⋅ ⋅)” indicates avariable differentiated twice with respect to time. Further, a matrixE(^) is a unit matrix, a vector g(→) is a vector indicating accelerationof gravity, and scalar m indicates a mass of the manipulation target M.

Similarly to the upper equation, the lower equation shows that momentmAe generated at the rotation center due to the external force acting onthe manipulation target M is represented by a component obtained bysubtracting a component p_g^t(→)×mR_w^t(^)g(→) of the moment due to thegravity from a component obtained by adding a component I(^)o_c^w(→)(⋅⋅)of torsional moment, a component o_c^w(→)(⋅)×I(^)o_c^w(→)(⋅) of momentdue to the Coriolis force, moment m^t(→) detected by the force sensor22, and a component p_e^t(→)×f^t(→) of moment by a component f^t(→) ofthe force detected by the force sensor 22.

Then, the load relaxation operation amount calculation unit 45calculates a small relaxation movement amount Δp^e(→) and a smallrelaxation rotation amount Δo^e(→) of the relaxation operation amount ofthe manipulation target M in the external force coordinate system Σ_eusing Equation (11) shown below, based on the force f^e(→) and themoment m^e(→) in the external force coordinate system Σ_e calculated instep S190 (step S200).

$\begin{matrix}{{\Delta\;{\overset{\;\rightarrow}{P}}^{e}} = {{\frac{1}{{{\hat{M}}_{p}\; s^{2}} + {{\hat{D}}_{p}\; s} + {\hat{K}}_{p}}\;{\overset{\;\rightarrow}{F}}^{e}\;\Delta\;{\overset{\;\rightarrow}{\; O}}^{e}} = {\frac{1}{{{\hat{M}}_{o}\; s^{2}} + {{\hat{D}}_{o}\; s} + {\hat{K}}_{o}}\;{\overset{\;\rightarrow}{M}}^{e}}}} & (11)\end{matrix}$

Here, ΔP^e(→) and ΔO^e(→) of Equation (11) used in step S200 areobtained by performing a Laplace transform on Δp^e(→) and Δo^e(→)respectively. Further, Equation (11) varies depending on a motion modelused to determine an operation amount (small movement amount and smallrotation amount) for movement in a direction in which the external forceis relaxed when the external force is applied to the rotation centerposition set in the manipulation target M. In this embodiment, as shownin Equation (11) described above, the motion model represented by aninertial mass matrix M_p(^), a damper coefficient matrix D_p(^), and aspring multiplier matrix K_p(^) regarding the small movement amount, andan inertial mass matrix M_o(^), a damper coefficient matrix D_o(^), anda spring multiplier matrix K_o(^) regarding the small rotation amount isadopted. The control performed by such a motion model is calledimpedance control among compliant motion controls. Further, the scalar sis a variable used for the Laplace transform. The compliant motioncontrol is not limited to impedance control, and for example, stiffnesscontrol or damping control may be applied.

Then, the load relaxation operation amount calculation unit 45calculates a small relaxation movement amount Δp_t^w(i)(→) and a smallrelaxation rotation amount Δo_t^w(i)(→) as the relaxation operationamount of the force sensor 22 in the work coordinate system Σ_w usingEquation (12) shown below, based on the relaxation operation amount ofthe manipulation target M in the external force coordinate system Σ_ecalculated in step S200 (step S210).Δ{right arrow over (p)}_(t) ^(w)(i)={circumflex over (R)}_(e)^(w)[Δ{right arrow over (p)}^(e)+Δ{right arrow over (o)}^(e)×p_(t) ^(e)]Δ{circumflex over (R)}_(t) ^(w)(i)={circumflex over (R)}_(e)^(w){circumflex over (R)}(Δ{right arrow over (o)}^(e)){circumflex over(R)}_(w) ^(e)  (12)

Here, “(i)” in the small relaxation movement amount Δp_t^w(i)(→) and thesmall relaxation rotation amount Δo_t^w(i)(→) that constitute therelaxation operation amount is a label for distinguishing between therelaxation operation amount and the target operation amount. The robotcontrol unit 49 of the control unit 36 can control the robot 20 to movethe position and the posture of the force sensor 22 to a position inwhich the force and the moment acting on the rotation center positionare relaxed, based on the relaxation operation amount in the workcoordinate system Σ_w calculated by the load relaxation operation amountcalculation unit 45 through the process from step S200 to step S210.

Then, the movement operation amount calculation unit 46 calculates atarget operation amount for moving the manipulation target M to thetarget position and the target posture, which is the small targetmovement amount Δp_m^c(→) and the small target rotation amount Δo_m^c(→)as the target operation amount in the imaging unit coordinate system Σ_cusing Equation (13) shown below, based on the target positionp_m^c(d)(→) and the target posture o_m^c(d)(→) of the manipulationtarget M in the imaging unit coordinate system Σ_c in a state in whichassembly of the manipulation target M and the assembly target O iscompleted, which is calculated in step S130, and the position p_(—)m^c(→) and the posture o_m^c(→) of the manipulation target M in theimaging unit coordinate system Σ_c detected in step S140 (step S220).Δ{right arrow over (p)}_(m) ^(c)(v)={circumflex over (K)}_(p)({rightarrow over (p)}_(m) ^(c)(d)−{right arrow over (p)}_(m) ^(c))Δ{right arrow over (o)}_(m) ^(c)(v)={circumflex over (K)}_(o){rightarrow over (o)}({circumflex over (R)}_(m) ^(c)(d)({circumflex over(R)}_(m) ^(c))⁻¹)  (13)

Then, the movement operation amount calculation unit 46 calculates thesmall target movement amount Δp_t^w(v)(→) and the small target rotationamount Δo_t^w (v)(→) as the target operation amount of the force sensor22 in the work coordinate system Σ_w using Equation (14) shown below,based on the target operation amount of the manipulation target M in theimaging unit coordinate system Σ_c calculated in step S220 (step S230).Δ{right arrow over (p)}_(t) ^(w)(v)={circumflex over (R)}_(e)^(w)└{circumflex over (R)}(Δ{right arrow over (o)}^(e)){circumflex over(R)}_(m) ^(e){{circumflex over (R)}_(c) ^(m)Δ{right arrow over (p)}_(m)^(c)(v)+{right arrow over (o)}({circumflex over (R)}_(c) ^(m){circumflexover (R)}(Δ{right arrow over (o)}_(m) ^(c)(v)){circumflex over (R)}_(m)^(c))×({right arrow over (p)}_(e) ^(m)+Δ{right arrow over(p)}^(e))}+{right arrow over (o)}({circumflex over (R)}(Δ{right arrowover (o)}^(e)){circumflex over (R)}_(m) ^(e){circumflex over (R)}_(c)^(m){circumflex over (R)}(Δ{right arrow over (o)}_(m)^(c)(v)){circumflex over (R)}_(m) ^(c){circumflex over (R)}_(e)^(m){circumflex over (R)}(Δ{right arrow over (o)}^(e))⁻¹)×{right arrowover (p)}_(t) ^(c)┘Δ{right arrow over (o)}_(t) ^(w)(v)={right arrow over (o)}(({circumflexover (R)}(Δ{right arrow over (o)}^(e)){circumflex over (R)}_(m)^(e))[{circumflex over (R)}(Δ{right arrow over (o)}^(e)){circumflex over(R)}_(m) ^(e){circumflex over (R)}_(c) ^(m){circumflex over(R)})(Δ{right arrow over (o)}_(m) ^(c)(v)){circumflex over (R)}_(m)^(c){circumflex over (R)}_(e) ^(m){circumflex over (R)}(Δ{right arrowover (o)}^(e))⁻¹]({circumflex over (R)}(Δ{right arrow over(o)}^(e)){circumflex over (R)}_(m) ^(e))⁻¹)  (14)

Here, “(v)” in the small target movement amount Δp_t^w(v)(→) and thesmall target rotation amount Δo_t ^w(v)(→) which constitute a targetoperation amount is a label for distinguishing between the targetoperation amount and the relaxation operation amount.

Then, the operation end determination unit 47 calculates an added smallmovement amount Δp_t^w (→) and an added small rotation amount Δo_t ^w(→) by adding the relaxation operation amount calculated in step S210and the target operation amount calculated in step S230 as in Equation(15) shown below (step S240).Δ{right arrow over (p)}_(t) ^(w)=Δ{right arrow over (p)}_(t)^(w)(i)+Δ{right arrow over (p)}_(t) ^(w)(v)Δ{circumflex over (R)}_(t) ^(w)={circumflex over (R)}(Δ{right arrow over(o)}_(t) ^(w)(v)){circumflex over (R)}(Δ{right arrow over (o)}_(t)^(w)(i))  (15)

Then, the operation end determination unit 47 determines whether both ofthe added small movement amount Δp_t^w(→) and the added small rotationamount Δo_t^w (→) calculated in step S240 are smaller than respectivecorresponding predetermined thresholds (step S250). Further, therespective corresponding predetermined thresholds are set separately.When both of the added small movement amount Δp_t^w(→) and the addedsmall rotation amount Δo_t^w(→) are smaller than the respectivecorresponding predetermined thresholds (step S250—Yes), the operationend determination unit 47 determines that it is no longer necessary tomove the manipulation target M, and ends the process.

On the other hand, when the operation end determination unit 47determines that one or both of the added small movement amount Δp_t^w(→)and the added small rotation amount Δo_t^w(→) are not smaller than therespective corresponding predetermined thresholds (step S250—No), therobot control unit 49 calculates the target position p_t^w (d)(→) andthe target posture o_t^w (d)(→) of the force sensor 22 in the workcoordinate system Σ_w using Equation (16) shown below, based on therelaxation operation amount calculated in step S210 and the targetoperation amount calculated in step S230 (step S260).{right arrow over (p)}_(t) ^(w)(d)={right arrow over (p)}_(t)^(w)+Δ{right arrow over (p)}_(t) ^(w){circumflex over (R)}_(t) ^(w)(d)={circumflex over (R)}_(t)^(w){circumflex over (R)}(Δ{right arrow over (o)}_(t) ^(w)  (16)

Further, the operation end determination unit 47 may be configured todetermine whether both of the small target movement amount Δp_t^w (v)(→)and the small target rotation amount Δo_t^w(v)(→) calculated in stepS230 are smaller than respective corresponding predetermined thresholds,instead of being configured to determine whether both of the added smallmovement amount Δp_t^w(→) and the added small rotation amount Δo_t^w (→)calculated in step S240 are smaller than respective correspondingpredetermined thresholds. Further, the operation end determination unit47 may be configured to determine whether a value obtained from anarbitrary function consisting of Δp_t^w(→)(i), Δo_t^w(i), Δp_t^w(→)(v)and Δo_t^w(v) is smaller than a predetermined threshold (whether it isno longer necessary to move the manipulation target M) using thefunction according to a property of predetermined work performed by therobot 20.

Then, the robot control unit 49 controls the robot 20 to move the forcesensor 22 so that the position and the posture of the force sensor 22match the target position p_t^w(d)(→) and the posture o_t^w(d)(→) of theforce sensor 22 in the work coordinate system Σ_w calculated in stepS260 (step S270). The control unit 36 repeats the processes of stepsS120 to S270 until it is determined in the determination of step S250that it is no longer necessary to move the manipulation target M, toperform control so that the robot 20 performs predetermined work.

Further, the rotation center position calculation unit 41 may include amarker indicating the position of the force sensor 22 described above,for example, when the force sensor 22 cannot be imaged by the imagingunit 10 because the force sensor 22 is inside the arm unit ARM and notvisible from the outside in the process from step S150 to step S180 ormay sequentially calculate the position p_t^e(→) and the postureo_t^e(→) of the force sensor 22 in the external force coordinate systemΣ_e using Equations (17) to (19) shown below, based on a relativepositional relationship in an initial state between the force sensor 22and the rotation center position and the rotation center posture set bya user.Δ{right arrow over (p)}_(m) ^(c)(t)={right arrow over (p)}_(m)^(c)(t)−{right arrow over (p)}_(m) ^(c)(t−1)−Δ{right arrow over (p)}_(m)^(c)(v)−└{circumflex over (R)}_(m) ^(c){circumflex over (R)}_(e)^(m){circumflex over (R)}(Δ{right arrow over (o)}^(e))┘{Δ{right arrowover (p)}^(e)(t)−Δ{right arrow over (p)}^(e)(t−1)+(Δ{right arrow over(o)}^(e)(t)−Δ{right arrow over (o)}^(e)(t−1))×{right arrow over (p)}_(m)^(e)}{circumflex over (R)}_(m) ^(c)(t)={{circumflex over (R)}({right arrowover (o)}_(m) ^(c)(v))⁻¹[{circumflex over (R)}_(m) ^(c){circumflex over(R)}_(e) ^(m){circumflex over (R)}(Δ{right arrow over(o)}^(e))]{circumflex over (R)}(Δ{right arrow over (o)}^(e)(t)−Δ{rightarrow over (o)}^(e)(t−1))[{circumflex over (R)}_(m) ^(c){circumflex over(R)}_(e) ^(m){circumflex over (R)}(Δ{right arrow over(o)}^(e))]⁻¹}⁻¹{circumflex over (R)}({right arrow over (o)}_(m)^(c)(t)−{right arrow over (o)}_(m) ^(c)(t−1))Δ{right arrow over (o)}_(m) ^(c)(t)={right arrow over (o)}({circumflexover (R)}_(m) ^(c)(t))  (17)Δ{right arrow over (p)}^(m)(t)={circumflex over (R)}_(c) ^(m)Δ{rightarrow over (p)}_(m) ^(c)(t)Δ{circumflex over (R)}^(m)(t)={circumflex over (R)}_(c) ^(m){circumflexover (R)}_(m) ^(c)(t){circumflex over (R)}_(m) ^(c)Δ{right arrow over (o)}^(m)(t)={right arrow over (o)}({circumflex over(R)}^(m)(t))  (18){right arrow over (p)}_(t) ^(e)(t)=[[{circumflex over (R)}(Δ{right arrowover (o)}^(e)({circumflex over (R)}_(m) ^(e)]{circumflex over(R)}^(m)(t)[{circumflex over (R)}(Δ{right arrow over(o)}^(e)){circumflex over (R)}_(m) ^(e)]⁻¹]⁻¹[{right arrow over (p)}_(t)^(e)(t−1)−{circumflex over (R)}(Δ{right arrow over (o)}^(e)){circumflexover (R)}_(m) ^(e)[Δ{right arrow over (p)}^(m)(t)+Δ{right arrow over(o)}^(m)(t)×({right arrow over (p)}_(e) ^(m)−Δ{right arrow over(p)}^(e))]]{circumflex over (R)}_(t) ^(e)(t)=[[{circumflex over (R)}(Δ{right arrowover (o)}^(e)){circumflex over (R)}_(m) ^(e)]{circumflex over(R)}^(m)(t)[{circumflex over (R)}(Δô^(e)){circumflex over (R)}_(m)^(e)]⁻¹]⁻¹{circumflex over (R)}_(t) ^(e)(t−1)){right arrow over (o)}_(t) ^(e)(t)={right arrow over (o)}({circumflexover (R)}_(t) ^(e)(t))  (19)

Equations (17) to (19) above are equations for calculating an amount ofshift of the relative positional relationship between the force sensor22 and the manipulation target M by subtracting a movement amount of therobot 20 from the movement amount of the manipulation target M detectedfrom the captured image. Further, the movement amount of the robot 20 iscalculated based on the initial state of the robot 20 by the controlunit 36. The configuration using the relative positional relationship inthe initial state of the force sensor 22 and the rotation centerposition and the rotation center posture set by the user, and Equations(17) to (19) above is useful because the position and the posture of themanipulation target M and the position and the posture of the forcesensor 22 can be tracked when the manipulation target M and the forcesensor 22 are not included in an imaging range of the imaging unit 10,that is, even when the position of the force sensor 22 cannot bedetected using the marker indicating the position of the force sensor22.

Even when the force sensor 22 cannot be imaged by the imaging unit 10,the rotation center position calculation unit 41 can perform the processof step S150 to step S180 by performing sequential calculation usingEquations (17) to (19) above, as in the case in which the force sensor22 can be imaged by the imaging unit 10. As a result, the control unit36 can control the robot 20 to perform predetermined work based on therelative positional relationship between the force sensor 22 and thepredetermined position set in the manipulation target M. Further, thecase in which the force sensor 22 cannot be imaged by the imaging unit10 is, for example, a case in which the force sensor 22 is covered witha member of the arm unit ARM or a case in which the arm unit ARMincluding the force sensor 22 is shifted from the angle of view of theimaging unit 10.

As described above, the robot 20 of the robot system 1 according to thefirst embodiment rotates the manipulation target M around the rotationcenter position of the manipulation target M and relatively moves themanipulation target M with respect to the assembly target O using thegripping unit HND based on the captured image including the grippingunit HND and the manipulation target M. Therefore, the robot can performgood assembly work.

Further, the robot 20 relatively moves the manipulation target M withrespect to the assembly target O through rotation and translation usingthe rotation center in the case of rotation as a coordinate originmoving with the manipulation target M. Therefore, the robot can performgood assembly work.

Further, the robot 20 can sequentially relatively move the manipulationtarget M with respect to the assembly target O, and thus, perform goodassembly work with high precision.

Further, the robot 20 performs compliant motion control according to amotion characteristic set in a predetermined position and each axialdirection. Therefore, the robot 20 can assemble the manipulation targetM with respect to the assembly target O without damaging the assemblytarget O.

Further, the robot 20 derives the relative positional relationshipbetween the position set in the manipulation target M and the positionset in the gripping unit HND based on the captured image, and updatesthe rotation center position based on the derived positionalrelationship. Therefore, even when the positional relationship of thegripping unit HND and the rotation center is shifted due to an externalforce, the robot 20 can relatively move the manipulation target M withrespective to the assembly target O around the shifted rotation centerand, as a result, perform good assembly work.

Further, the robot 20 derives a relative positional relationship betweenthe position set in the gripping unit HND and the position set in themanipulation target M, and updates the rotation center position based onthe derived positional relationship and the relative positionalrelationship between the position set in the manipulation target M andthe rotation center position. Therefore, even when the positionalrelationship between the gripping unit HND and the rotation centerposition is shifted due to an external force, the robot 20 canindirectly recognize the shifted rotation center position from therelative positional relationship between the position set in thegripping unit HND and the rotation center position through the positionset in the manipulation target M and, as a result, can relatively movethe manipulation target M with respect to the assembly target O aroundthe shifted rotation center position.

Further, the robot 20 derives the relative positional relationshipbetween the position set in the gripping unit HND and the rotationcenter position based on the position of the marker indicating theposition of the force sensor 22 detected from the captured image, andupdates the rotation center position based on the relative positionalrelationship between the position set in the gripping unit HND and therotation center position. Therefore, even when the force sensor 22 iscovered with a member of the arm unit ARM of the robot 20, the robot 20can recognize the position of the manipulation target M using theposition of the marker indicating the force sensor 22 as a mark and, asa result, can perform good assembly work.

Further, the robot 20 derives the relative positional relationshipbetween the position set in the gripping unit HND and the rotationcenter position based on the position and the posture in the initialstate of the force sensor 22, and updates the rotation center positionbased on the derived relative positional relationship between theposition set in the gripping unit HND and the rotation center position.Therefore, even when the arm unit ARM of the robot 20 including theforce sensor 22 is shifted from an angle of view of the imaging unit 10,the robot 20 can recognize the position of the manipulation target Mand, as a result, can perform good assembly work.

Second Embodiment

Hereinafter, a second embodiment of the present invention will bedescribed with reference to the drawings. The robot system 2 accordingto the second embodiment may have a configuration in which a dual-armrobot is included as the robot 25, in place of the configuration inwhich the single arm robot is included as the robot 20. Further, in thesecond embodiment, the same constituent units as those in the firstembodiment are denoted with the same reference signs.

FIG. 6 is a configuration diagram illustrating an example of the robotsystem 2 according to the second embodiment. The robot system 2 includesan imaging unit 10, a robot 25, and a control device 30.

In the second embodiment, an assembly target O is installed on a standsuch as a table by a jig or the like, and a manipulation target M isassembled through the predetermined work described in the firstembodiment, by any one arm of the robot 25 which is a dual-arm robot.Further, the assembly target O may be gripped by a gripping unit HND2 ofthe robot 25 and the predetermined work described in the firstembodiment may be performed by a gripping unit HND1. Further, in thiscase, a role of the gripping unit HND1 and the gripping unit HND2 may bereversed.

The robot 25 is, for example, a dual-arm robot in which the grippingunit HND1, the gripping unit HND2, a force sensor 22, an arm unit ARM1,an arm unit ARM2, and a plurality of actuators, which are notillustrated, are included in each arm, as illustrated in FIG. 6.

Each arm of the robot 25 is a 6-axis vertical multi-joint type. One armcan perform an operation with degrees of freedom defined by six axesthrough an operation in which the support stand, the arm unit ARM1, andthe gripping unit HND1 cooperate using the actuator, and the other armcan perform an operation with degrees of freedom defined by six axesthrough an operation in which the support stand, the arm unit ARM2, andthe gripping unit HND2 cooperate using the actuator. Further, each armof the robot 20 may operate with 5 degrees of freedom (on 5 axes) orless or may operate with 7 degrees of freedom (on 7 axes) or more.

While the robot 25 performs an operation controlled by the controldevice 30 described in the first embodiment using the arm including thegripping unit HND1 and the arm unit ARM1, the same operation may beperformed using the arm including the gripping unit HND2 and the armunit ARM2. Further, each of the gripping unit HND1 and the gripping unitHND2 is an example of a hand. The robot 25 is connected to the controldevice 30, for example, by a cable so that the robot 25 can communicatewith the control device 30. Wired communication via the cable isperformed according to, for example, a standard such as Ethernet(registered trademark) or USB. Further, the robot 25 and the controldevice 30 may be connected through wireless communication performedaccording to a communication standard such as Wi-Fi (registeredtrademark).

Further, the robot 25 is controlled by the control device 30 mountedinside the robot 25 as illustrated in FIG. 6, the control device 30 maybe installed outside the robot 25, in place of such a configuration.

As described above, since the robot 25 of the robot system 2 accordingto the second embodiment is the dual-arm robot and the predeterminedwork described in the first embodiment is performed by either or both ofthe two arms of the dual-arm robot, it is possible to obtain the sameeffects as in the first embodiment.

Third Embodiment

Hereinafter, a third embodiment of the present invention will bedescribed with reference to the drawings. Further, in the thirdembodiment, the same constituent units as in the first embodiment aredenoted with the same reference signs. In predetermined work, a robotsystem 1 according to the third embodiment assembles, for example, awrench (an example of a manipulation target M) gripped by a grippingunit HND and a bolt (an example of an assembly target O), and thentightens the bolt with the gripped wrench.

Here, the predetermined work performed by the robot system 1 accordingto the third embodiment will be described with reference to FIGS. 7A to7D. FIGS. 7A to 7D are diagrams schematically illustrating an example ofthe predetermined work performed by the robot system 1 according to thethird embodiment. FIG. 7A illustrates a state of a predetermined initialposition before a wrench M1 gripped by the robot 20 is assembled with abolt O1. The robot system 1 moves the wrench M1 to a position (aposition immediately before the bolt O1) of a wrench VM1 shown by atwo-dot chain line in FIG. 7A. Here, an image showing a state of thewrench VM1 is set in the robot system 1 in advance, and the robot system1 moves the gripping unit HND through the process described withreference to FIG. 5 in the first embodiment so as to realize such astate. A state in which the wrench M1 is assembled with the bolt O1 bythe robot 20 is illustrated in FIG. 7B. The robot system 1 moves thegripping unit HND from the state of the wrench VM1 illustrated in FIG.7A to the state in which the wrench M1 and the bolt O1 are assembled,which is illustrated in FIG. 7B, through the process described withreference to FIG. 5. Here, an image showing the wrench M1 and the boltO1 in the state illustrated in FIG. 7B is set in the robot system 1 inadvance, and the robot system 1 moves the gripping unit HND through theprocess described with reference to FIG. 5 in the first embodiment so asto realize such a state.

In FIG. 7C, a state in which the bolt O1 is rotated 60° by the wrench M1gripped by the robot 20 is illustrated. An image showing a rotated state(for example, a state of a wrench VM2 and a bolt VO1 indicated by atwo-dot chain line in FIG. 7C) is set in the robot system 1 in advance,and the robot system 1 moves the gripping unit HND based on the processdescribed with reference to FIG. 5 so as to realize such a state.Further, the bolt O1 and the bolt VO1 are shown in an overlapped mannerin FIG. 7B due to their regular hexagons.

In FIG. 7D, a state in which the bolt O1 is rotated by the wrench M1gripped by the robot 20 is illustrated. When the bolt O1 is rotated bythe wrench M1 illustrated in FIG. 7D, the robot system 1 moves thewrench M1 to a position apart by a predetermined distance from the boltO1. Here, an image showing a state in which the wrench M1 moves to theposition apart by a predetermined distance from the bolt O1 is set inthe robot system 1 in advance, and the robot system 1 moves the grippingunit HND through the process described with reference to FIG. 5 in thefirst embodiment so as to realize such a state. Also, the robot system 1moves the gripping unit HND until the wrench M1 moves from the positionapart by the predetermined distance from the bolt O1 to the initialposition. The robot system 1 tightens the bolt O1 with the wrench M1 byrepeating the process of performing the operations illustrated in FIGS.7A to 7D.

Hereinafter, a process in which the control unit 36 of the controldevice 30 operates the robot 20 to tighten the bolt O1 with the wrenchM1 will be described with reference to FIG. 8. FIG. 8 is a flowchartillustrating an example of a process flow in which the control unit 36of the control device 30 operates the robot 20 to tighten the bolt O1with the wrench M1.

In the following description, the robot system 1 is assumed to havealready performed the process from steps S100 to S110 illustrated inFIG. 5. First, the control unit 36 moves the gripping unit HND of therobot 20 until the wrench M1 gripped by the gripping unit HND isarranged in the predetermined initial position described above in aninitial arrangement process (step S300). Here, the initial arrangementprocess refers to a process from step S120 to step S270 illustrated inFIG. 5.

Then, the control unit 36 assembles the wrench M1 and the bolt O1through an assembling process (step S310). Here, the assembling processrefers to a process of moving the gripping unit HND until the wrench M1and the bolt O1 enter the state illustrated in FIG. 7B and assemblingthe wrench M1 with the bolt O1 through the process from step S120 tostep S270 illustrated in FIG. 5.

Then, the control unit 36 gets ready to move the gripping unit HNDgripping the wrench M1 until the wrench M1 and the bolt O1 enter thestate illustrated in FIG. 7D through a bolt tightening preparationprocess (step S320). Here, the bolt tightening preparation processrefers to a process of calculating an added small movement amount and anadded small rotation amount for moving the gripping unit HND androtating the bolt O1 with the wrench M1 through the process from stepS120 to step S240 illustrated in FIG. 5.

Then, the control unit 36 determines whether tightening of the bolt O1is completed by determining whether the moment (corresponding totightening torque) calculated using Equation (10) described above instep S320 is equal to or greater than a predetermined value (step S330).When the control unit 36 determines that the calculated moment is equalto or greater than the predetermined value (step S330—Yes), the controlunit 36 determines that tightening of the bolt O1 is completed, and endsthe process. On the other hand, when the control unit 36 determines thatthe calculated moment is smaller than the predetermined value (stepS330—No), the control unit 36 determines whether both of the added smallmovement amount and the added small rotation amount calculated usingEquation (15) described above in step S320 are smaller thancorresponding predetermined thresholds (step S340).

When the control unit 36 determines that both of the added smallmovement amount and the added small rotation amount are equal to orgreater than the corresponding predetermined thresholds (step S340—No),the control unit 36 determines that the wrench M1 and the bolt O1 do notreach the state illustrated in FIG. 7D, and calculates the targetposition and posture of the force sensor 22 in the work coordinatesystem Σ_w based on the relaxation operation amount and the targetoperation amount calculated in step S320 (step S360). Then, the controlunit 36 further rotates the bolt O1 with the wrench M1 by controllingthe robot 20 to move the force sensor 22 so that the position and theposture of the force sensor 22 match the target position and the targetposture of the force sensor 22 in the work coordinate system Σ_wcalculated in step S360 (step S370). Further, since step S360 is thesame process as step S260 and step S370 is the same process as stepS270, detailed description thereof will be omitted. On the other hand,when the control unit 36 determines that both of the added smallmovement amount and the added small rotation amount are smaller than thecorresponding predetermined thresholds (step S340—Yes), the control unit36 determines that the wrench M1 and the bolt O1 have reached the stateillustrated in FIG. 7D, releases the wrench M1 from the bolt O1 (stepS350), and then, performs the process of step S300.

As described above, based on the process described in FIG. 5 in thefirst embodiment and the process of step S330 according to the momentcorresponding to the tightening torque, the robot system 1 according tothe third embodiment can combine control based on visual servo andcompliant motion control based on a force sensor, and reliably tightenthe bolt O1 with the wrench M1 gripped by the gripping unit HND.

Further, a program for realizing the functions of any constituent unitsin the above-described device (for example, the robot system 1 or 2) maybe recorded in a computer-readable recording medium and loaded into andexecuted by a computer system. Further, the “computer system” referredto herein includes an OS (Operating System) or hardware such as aperipheral device. Further, the “computer-readable recording medium”includes a flexible disk, a magnetic optical disc, a ROM (Read OnlyMemory), a portable medium such as a CD (Compact Disk)-ROM, or a storagedevice such as a hard disk built in the computer system. Further, the“computer-readable recording medium” also includes a recording mediumthat holds a program for a certain time, such as a volatile memory (RAM:Random Access Memory) inside a computer system including a server and aclient when a program is transmitted via a network such as the Internetor a communication line such as a telephone line.

Further, the above-described program may be transmitted from a computersystem in which the program is stored in a storage device or the like toother computer systems via a transmission medium or by transmissionwaves in the transmission medium. Here, the “transmission medium” fortransmitting the program refers to a medium having a function oftransmitting information, such as a network (communication network) suchas the Internet or a communication line such as a telephone line.

Also, the above-described program may be a program for realizing some ofthe above-described functions. Alternatively, the program may be aprogram capable of realizing the above-described functions incombination with a program previously stored in a computer system, thatis, a differential file (a differential program).

While preferred embodiments of the invention have been described andillustrated above, it should be understood that these are exemplary ofthe invention and are not to be considered as limiting. Additions,omissions, substitutions, and other modifications can be made withoutdeparting from the scope of the present invention. Accordingly, theinvention is not to be considered as being limited by the foregoingdescription, and is only limited by the scope of the appended claims.

What is claimed is:
 1. A robot comprising: a hand; and a control unitconfigured to operate the hand, wherein the control unit is configuredto move a first object toward a second object and is configured tooperate the hand to control a position and a posture of the first objectrelative to a predetermined position based on a captured image includingthe hand and the first object, and wherein the control unit includes: arotation center position calculation unit that detects a currentposition and a current posture of the first object, and calculates arotation center position and a rotation center posture based on thecurrent position and the current posture of the first object; a targetderivation unit that detects a position and a posture of the secondobject based on the captured image acquired by an image acquisition unitand determines a target position and a target posture of the firstobject based on the captured image and the position and the posture ofthe second object; a movement operation amount calculation unit thatdetermines a target operation amount of the first object, the targetoperation amount being an amount the first object is moved by the handto match the target position and the target posture; and a robot controlunit that calculates a target position and a target posture to which aforce sensor is to move based on the target operation amount of thefirst object and moves the force sensor by way of the hand so that theposition and the posture of the force sensor match the target positionand the target posture of the force sensor.
 2. The robot according toclaim 1, wherein the predetermined position is a coordinate origin thatmoves with the first object, and the control unit is configured totranslate and rotate the first object.
 3. The robot according to claim1, wherein the control unit is configured to perform visual servocontrol based on the captured image.
 4. The robot according to claim 1,wherein the control unit is configured to perform compliant motioncontrol according to a motion characteristic set in the predeterminedposition and each axial direction.
 5. The robot according to claim 1,wherein the control unit is configured to derive a relative positionalrelationship between a position set in the hand and a position set inthe first object based on the captured image, and update thepredetermined position based on the derived positional relationship. 6.The robot according to claim 5, wherein the control unit is configuredto update the predetermined position based on the derived positionalrelationship, and a relative positional relationship between theposition set in the first object and the predetermined position.
 7. Therobot according to claim 1, further comprising: a marker indicating aposition set in the hand, wherein the captured image further includesthe marker, and the control unit is configured to derive a relativepositional relationship between a position set in the hand and thepredetermined position based on the position of the marker detected fromthe captured image, and update the predetermined position based on thederived relative positional relationship between the position set in thehand and the predetermined position.
 8. The robot according to claim 7,wherein, if the control unit detects the marker and the position and theposture of the first object, the control unit is configured to updatethe predetermined position based on a first captured image including theimaged marker, a second captured image including the imaged firstobject, and a relative positional relationship between a first cameracapturing the first captured image and a second camera capturing thesecond captured image.
 9. The robot according to claim 1, wherein thecontrol unit is configured to derive a relative positional relationshipbetween the position set in the hand and the predetermined positionbased on the position in an initial state set in the hand, and updatethe predetermined position based on the derived relative positionalrelationship between the position set in the hand and the predeterminedposition.
 10. A robot system comprising: a robot including a hand; andan imaging unit that images the hand and a first object, wherein therobot is configured to move the first object toward a second object andis configured to operate the hand to control a position and a posture ofthe first object relative to a predetermined position based on acaptured image including the hand and the first object captured by theimaging unit, and wherein the robot includes: a rotation center positioncalculation unit that detects a current position and a current postureof the first object, and calculates a rotation center position and arotation center posture based on the current position and the currentposture of the first object; a target derivation unit that detects aposition and a posture of the second object based on the captured imageacquired by an image acquisition unit and determines a target positionand a target posture of the first object based on the captured image andthe position and the posture of the second object; a movement operationamount calculation unit that determines a target operation amount of thefirst object, the target operation amount being an amount the firstobject is moved by the hand to match the target position and the targetposture; and a robot control unit that calculates a target position anda target posture to which a force sensor is to move based on the targetoperation amount of the first object and moves the force sensor by wayof the hand so that the position and the posture of the force sensormatch the target position and the target posture of the force sensor.11. A control device for operating a robot including a hand, wherein thecontrol device is configured to cause the robot to move a first objecttoward a second object and is configured to operate the hand to controla position and a posture of the first object relative to a predeterminedposition based on a captured image including the hand and the firstobject, and wherein the control device includes: a rotation centerposition calculation unit that detects a current position and a currentposture of the first object, and calculates a rotation center positionand a rotation center posture based on the current position and thecurrent posture of the first object; a target derivation unit thatdetects a position and a posture of the second object based on thecaptured image acquired by an image acquisition unit and determines atarget position and a target posture of the first object based on thecaptured image and the position and the posture of the second object; amovement operation amount calculation unit that determines a targetoperation amount of the first object, the target operation amount beingan amount the first object is moved by the hand to match the targetposition and the target posture; and a robot control unit thatcalculates a target position and a target posture to which a forcesensor is to move based on the target operation amount of the firstobject and moves the force sensor by way of the hand so that theposition and the posture of the force sensor match the target positionand the target posture of the force sensor.
 12. A control method foroperating a robot including a hand, comprising: acquiring a capturedimage including the hand and a first object; detecting a currentposition and a current posture of the first object, and calculating arotation center position and a rotation center posture based on thecurrent position and the current posture of the first object; detectinga position and a posture of the second object based on the capturedimage acquired by an image acquisition unit and determining a targetposition and a target posture of the first object based on the capturedimage and the position and the posture of the second object; determininga target operation amount of the first object, the target operationamount being an amount the first object is moved by the hand to matchthe target position and the target posture; calculating a targetposition and a target posture to which a force sensor is to move basedon the target operation amount of the first object and moving the forcesensor by way of the hand so that the position and the posture of theforce sensor match the target position and the target posture of theforce sensor; and moving the first object toward a second object andoperating the hand to control a position and a posture of the firstobject relative to a predetermined position based on the captured image,the rotation center position and the rotation center posture.